Rates of convergence for the posterior distributions of mixtures of Betas and adaptive nonparametric estimation of the density.

In this paper, we investigate the asymptotic properties of nonparametric Bayesian mixtures of Betas for estimating a smooth density on [0, 1]. We consider a parametrization of Beta distributions in terms of mean and scale parameters and construct a mixture of these Betas in the mean parameter, while putting a prior on this scaling parameter. We prove that such Bayesian nonparametric models have good frequentist asymptotic properties. We determine the posterior rate of concentration around the true density and prove that it is the minimax rate of concentration when the true density belongs to a Hölder class with regularity β, for all positive β, leading to a minimax adaptive estimating procedure of the density. We also believe that the approximating results obtained on these mixtures of Beta densities can be of interest in a frequentist framework.kernel; Bayesian nonparametric; rates of convergence; mixtures of Betas; adaptive estimation;

About the posterior distribution in hidden Markov models with unknown number of states

We consider finite state space stationary hidden Markov models (HMMs) in the situation where the number of hidden states is unknown. We provide a frequentist asymptotic evaluation of Bayesian analysis methods. Our main result gives posterior concentration rates for the marginal densities, that is for the density of a fixed number of consecutive observations. Using conditions on the prior, we are then able to define a consistent Bayesian estimator of the number of hidden states. It is known that the likelihood ratio test statistic for overfitted HMMs has a nonstandard behaviour and is unbounded. Our conditions on the prior may be seen as a way to penalize parameters to avoid this phenomenon. Inference of parameters is a much more difficult task than inference of marginal densities, we still provide a precise description of the situation when the observations are i.i.d. and we allow for $2$ possible hidden states.Comment: Published in at http://dx.doi.org/10.3150/13-BEJ550 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Non parametric finite translation mixtures with dependent regime

In this paper we consider non parametric finite translation mixtures. We prove that all the parameters of the model are identifiable as soon as the matrix that defines the joint distribution of two consecutive latent variables is non singular and the translation parameters are distinct. Under this assumption, we provide a consistent estimator of the number of populations, of the translation parameters and of the distribution of two consecutive latent variables, which we prove to be asymptotically normally distributed under mild dependency assumptions. We propose a non parametric estimator of the unknown translated density. In case the latent variables form a Markov chain (Hidden Markov models), we prove an oracle inequality leading to the fact that this estimator is minimax adaptive over regularity classes of densities

Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator

We consider the asymptotic behaviour of the marginal maximum likelihood empirical Bayes posterior distribution in general setting. First we characterize the set where the maximum marginal likelihood estimator is located with high probability. Then we provide oracle type of upper and lower bounds for the contraction rates of the empirical Bayes posterior. We also show that the hierarchical Bayes posterior achieves the same contraction rate as the maximum marginal likelihood empirical Bayes posterior. We demonstrate the applicability of our general results for various models and prior distributions by deriving upper and lower bounds for the contraction rates of the corresponding empirical and hierarchical Bayes posterior distributions.Comment: 36 pages +24 pages supplementary materia

Rates of convergence for the posterior distributions of mixtures of Betas and adaptive nonparametric estimation of the density

In this paper, we investigate the asymptotic properties of nonparametric Bayesian mixtures of Betas for estimating a smooth density on $[0,1]$. We consider a parametrization of Beta distributions in terms of mean and scale parameters and construct a mixture of these Betas in the mean parameter, while putting a prior on this scaling parameter. We prove that such Bayesian nonparametric models have good frequentist asymptotic properties. We determine the posterior rate of concentration around the true density and prove that it is the minimax rate of concentration when the true density belongs to a H\"{o}lder class with regularity $\beta$, for all positive $\beta$, leading to a minimax adaptive estimating procedure of the density. We also believe that the approximating results obtained on these mixtures of Beta densities can be of interest in a frequentist framework.Comment: Published in at http://dx.doi.org/10.1214/09-AOS703 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Studentization and the determination of p-values.

The original Studentization was the conversion of a sample mean departure into the familiar t-statistic, plus the derivation of the corresponding Student distribution function; the observed value of the distribution function is the observed p-value, as presented in an elemental form. We examine this process in a broadly general context: a null statistical model is available together with observed data; a statistic t(y) has been proposed as a plausible measure of the location of the data relative to what is expected under the null; a modified statistic, say ~t(y), is developed that is ancillary; the corresponding distribution function is determined, exactly or approximately; and the observed value of the distribution function is the p-value or percentage position of the data with respect to the model. Such p-values have had extensive coverage in the recent Bayesian literature, with many variations and some preference for two versions labelled pppost and pcpred. The bootstrap method also directly addresses this Studentization process. We use recent likelihood theory that gives a third order factorization of a regular statistical model into a marginal density for a full dimensional ancillary and a conditional density for the maximum likelihood variable. The full dimensional ancillary is shown to lead to an explicit determination of the Studentized version ~t(y) together with a highly accurate approximation to its distribution function; the observed value of the distribution function is the p-value and is available numerically by direct calculation or by Markov chain Monte Carlo or by other simulations. In this paper, for any given initial or trial test statistic proposed as a location indicator for a data point, we develop: an ancillary based p-value designated panc; a special version of the Bayesian pcpred; and a bootstrap based p-value designated pbs. We then show under moderate regularity that these are equivalent to the third order and have uniqueness as a determination of the statistical location of the data point, as of course derived from the initial location measure. We also show that these p-values have a uniform distribution to third order, as based on calculations in the moderate-deviations region. For implementation the Bayesian and likelihood procedures would perhaps require the same numerical computations, while the bootstrap would require a magnitude more in computation and would perhaps not be accessible. Examples are given to indicate the ease and exibility of the approachAncillary; Bayesian; Bootstrap; Conditioning; Departure measure; Likelihood; p-value; Studentization.;

Asymptotic behaviour of the posterior distribution in overfitted mixture models.

In this paper we study the asymptotic behaviour of the posterior distribution in a mixture model when the number of components in the mixture is larger than the true number of components, a situation commonly referred to as overfitted mixture. We prove in particular that quite generally the posterior distribution has a stable and interesting behaviour, since it tends to empty the extra components. This stability is achieved under some restriction on the prior, which can be used as a guideline for choosing the prior. Some simulations are presented to illustrate this behaviour.posterior concentration; mixture models; overfitting; Asymptotic; Bayesian;